#718 billion parameters

Huawei has introduced Pangu Ultra MoE, a 718 billion parameter sparse language model optimized for Ascend NPUs using simulation-driven architecture and advanced system-level optimizations to achieve high efficiency and performance.